1. Field of the Invention
Embodiments of the present invention generally relate to file system and storage management and, more particularly, to a method and apparatus for managing and optimizing storage space allocation for computer data in disk file systems and distributed file systems.
2. Description of the Related Art
Currently, an operating system temporarily stores input/output activity (e.g., system calls, such as create file, delete file, write data and/or the like) in computer memory (e.g., RAM) until flush time. After each system call, the operating system completes various file data and metadata allocation decisions that define storage space for the computer data. Subsequently, the operating system flushes information regarding storage and/or file system operations that update the file data and metadata in accordance with such storage space allocation decisions. In other words, the one or more operating systems periodically copy the storage and/or file system operations to hard disk.
The hard disk may include computer data (e.g., files and directories) that is organized into a file system, such as File Allocation Table (FAT), New Technology File System (NTFS). Generally, a file system is a method of organizing and storing computer files. The file system may be used to retrieve file data from a plurality of storage devices. For example, the DOS, WINDOWS, OS/2, MACINTOSH and UNIX based operating systems all have file systems in which files are placed somewhere in a hierarchical (tree) structure. A file is placed in a directory (folder in Windows) or subdirectory at the desired place in the tree structure. File systems may use a data storage device, such as a hard disk or CD-ROM, to maintain the physical location of the file data.
The computer data may be organized in accordance with a transactional file system, such as Transactional NTFS. Generally, the computer data in a transactional file system volume is fault-tolerant and consistent. A transaction can either be finished completely (e.g., a committed transaction) or reverted completely (e.g., a rolled back transaction), but not necessarily both at any given point in time. This means that if there is a crash or power failure, after recovery, the storage state of the computer data will be consistent. A significant amount of computing overhead, however, is required to maintain data consistency.
Unfortunately, interaction with certain disk file systems causes a significant amount of random input/output activity that is replete with interdependencies between various storage and/or file system operations. For example, metadata cannot be flushed until log records are finished being written into the log. As another, metadata updates to various bit maps have to be performed in an appropriate order when such metadata updates affect overlapping portions of the computer data. In addition, log-based file systems (e.g., EXT3) do not reclaim storage space after file deletions. Furthermore, metadata may be inconsistent in block-based file systems (e.g., SUN file system, UNIX file system).
Currently, disk file systems and distributed file system allocate storage space (e.g., hard disk space) for file data updates and metadata updates at system call time. For example, when data is written into a file, which causes the file to grow, such file systems determine a portion of a hard disk for the data to reside. Thus, for each and every write operation, a storage space allocation decision is made. Similarly, when other system calls are made (e.g., file or directory creation), the metadata updates are also determined (e.g., directory block allocation decision) at system call time. For example, file creation results in an allocation of an iNode as well as an allocation of a disk block to store a file name and an iNode number within a parent directory data area.
Some file systems journal each and every metadata updates, such that writes to the journal log are done as a result of each one of these file data space and metadata space allocation decisions. Unfortunately, the operating system responds to system calls with immediate space allocation decisions even if one or more files are subsequently deleted. Because the space allocation decisions have already been made, any effects must be undone. For example, one or more additional metadata updates may be journaled to reflect availability of one or more portions of the metadata (e.g., blocks and iNodes that are now free). In order to reduce a number of storage space allocations, current operating systems may implement various optimization tricks. For example, the operating system over allocates storage space at an end of a file with an expectation that future input/output activity (e.g., subsequent writes) consume that over allocated storage space.
Therefore, there is a need in the art for a method and apparatus for optimizing storage space allocations, using at least one processor, for computer data in disk file systems and distributed file systems.